Educational Software Cooperative 4

home *** CD-ROM | disk | FTP | other *** search

/ Educational Software Cooperative 4 / Educational Software Cooperative 4.iso / truemd23 / manual.doc < prev next >

Wrap

Text File | 1995-12-20 | 31.3 KB | 738 lines

------------------------------------------------------------ TRUE MULTIPLE-CHOICE INSTRUCTIONS 0/13 INDEX ------------------------------------------------------------ REVIEW, GAME and TEST Cram, Practice and Test . . . . . . . . . 1 Levels of Thinking . . . . . . . . . . . 2 Teacher Talk . . . . . . . . . . . . . . 3 Test Record Sample . . . . . . . . . . . 4 ------------------------------------------------------------ WRITE and EDIT Create and Enter Questions . . . . . . . 5 Select and Edit Questions . . . . . . . . 6 Print Cram Notes and Files . . . . . . . 7 Format and Print Tests . . . . . . . . . 8 ------------------------------------------------------------ CRITICAL READING WITH QUESTION WRITING Instructions . . . . . . . . . . . . . . 9 Question Form . . . . . . . . . . . . . 10 Problems . . . . . . . . . . . . . . . . 11 ------------------------------------------------------------ BIBLIOGRAPHY Asking the Questions (1980-1990) . . . . 12 The Answers (1991-1994) . . . . . . . . . 12 Application in the Classroom (1995-) . . 13 Quality Management (TQM, CQI) . . . . . . 13 ------------------------------------------------------------ Cram, Practice and Test 1/13 Questions are presented at five levels of thinking. The lowest level is random guessing by the computer. The highest level is reporting key terms to the computer without first having seen the answer options. The answer options do not have to be memorized. Only key terms, or significant parts of key terms, need be entered as answers (a minimum of five characters). The program has three modes: Tourists cram, Pupils practice, and Students report (or test) their understanding: 1. CRAM is a SEQUENTIAL presentation of questions in a rational order. Items answered incorrectly are repeated. They must be answered twice correctly to end the repetition. The goal is to familiarize the Tourist, to complete a review of the assignment in a reasonable amount of time. 2. PRACTICE is a RANDOM presentation of questions. It is a practice in recognizing questions as a sample of the body of related questions. Items answered incorrectly are repeated. The goal is to practice self-judgment, to pass and get a correct answer rather than to guess randomly. 3. TEST is like practice except items answered incorrectly are not repeated. Now is the time for the Student to report what is known or can be reasoned and to do so at the highest levels of thinking. The level of responsibility for knowing when a correct answer is entered (self-judgment) is set at 50% for CRAM and PRACTICE. You can elect three levels when taking a TEST: Pupil, 50%; Student, 70%; and Professional, 90%. The level of 50% is operational at all levels of thinking. The level of 90% may appear low when related to the professional activities of nurses or pilots, however, few tests contain a set of questions in which 90% perform well. Review, Game and Test has three goals: 1. To help you survive until you can question and learn without its help. 2. To help you abandon the passive concrete level of thinking and discover the active self-correcting level of thinking needed to understand and do science (and other things) in school and college. 3. To help build your confidence to the point you can write a record of your questions as you make sense of an assignment: a review question file. The Write and Edit portion of this program will help you do this alone or with your classmates. Levels of Thinking 2/13 Each question is presented at the highest level of thinking. If you cannot answer, press the "ENTER" key for the next lower level of thinking. At the bottom, one more press will display a right answer during CRAM and PRACTICE. Student: (2c Higher Level) Top students prefer to enter their own answers rather than select from multiple choice options. Your own answer needs to be short and unique. Something that is a part of a right answer but not of a wrong answer. The entry must be a minimum of five characters long, or the complete right answer if shorter. The query, biolo, will find biology and biological. The query, water AND hot, will find hot water but not water alone. The query, 1 OR one OR first, will find an answer in three different ways. You do not need to memorize the answers. You do need to classify and think of alternative answers. Scholar: (2b Higher Level) This produces startling effects on first time users. The question vanishes as the answers appear. You must have read the question with sufficient understanding to answer without referring back to the question. This is very much like an oral examination. Practice visualizing the question. Scientist: (2a Higher Level) Enter a false answer. It will be erased. Continue until only one answer remains, the true answer. Falsifying answers is a powerful way to study. Rather than having one key term related to one question stem, you will know several key terms that can be the correct answers to several questions. Pupil: (1 Concrete Level) Marking the correct answer option is the traditional way of taking multiple-choice tests. Here you enter an answer only if you are confident it is right. Tourist: (0 Lower Level) If you cannot recall or reason an answer, good judgment is to pass. An acceptable answer will appear during CRAM and PRACTICE. Consider why it is a correct answer before continuing. You may need to review class notes, do more reading, make more observations, or try a different line of reasoning. Teacher Talk 3/13 The BEST TEST REPORT is obtained by reading through the question file at least three times. Each time select more difficult items: 1. Answer with confidence. 2. Use easy educated guessing. 3. Use hard educated guessing. 4. Use random guessing (on a forced-choice test). Press PROGRESS for a PROGRESS REPORT at the end of each pass through the file or after every 10 or so questions answered. The information is presented in graphs and in tables. If you are making good progress, keep on selecting more questions to answer. If not, you need to stop. The NEW PROGRESS REPORT at the end of the test will print to the screen, to a printer, or to a file. If you need the record, then you must print to a printer or to a file (NEWPROG.REC). A Test Record sample is on the next page. This person did very well on the first 40 questions. With good self-judgment, she/he should have stopped reporting. With poor self-judgment, or on a forced-choice test, the remaining ten questions were answered by random guessing. The OLD REPORT RECORDS file keeps general information on all three modes: CRAM, PRACTICE, and TEST. The records are kept in the OLDREPRT.REC file. This is a permanent file that will print to the screen or to a printer. The Report Label does not show on the screen. It does print out on the printer. Students can see how others have done but cannot see who did the work. GAMES scoring erases all low quality test scores and all CRAM and PRACTICE scores. A STANDARD TEST consists of 50 questions. If more are in a file than 50, then 50 will be randomly selected. The PERCENT RIGHT is always correct. The SCORE is valid only with a file of 50 or more questions. The AVERAGE LEVEL OF THINKING for the entire review or test is calculated on a scale of 0 to 2, Tourist to Student. In general, all multiple/choice questions yield one Student point when answered correctly at the Student, Scholar, or Scientist levels. Short answer and true/false yield 0.5 point Student and Pupil each. All wrong answers yield one Tourist point. On a scale of zero for Tourist to two for Student, any score related to a level of thinking below one (Pupil) is of little value. At 1.7 empirical research has shown that students are actively reporting to the computer rather than selecting from answers. Other students in the class will accept a grade with this quality level as an indicator that the student really knows or can do something and is not just playing the traditional cram and guess game that is a part of forced-choice testing (teach, test, and rank). Test Record Sample 4/13 Report for: Sample Report Question File: SFAA4.IQF at 19:07:50 on 09-21-1993 SCORING SEQUENCE WITH 31 RIGHT AND 19 WRONG ---------------------------------------------------------- . . . . . . . 2 . . . . . 2 2 2 . . 70* S T A R T I N G . R E C I T I2N G2 2 . 70% . (Student) . . S E N S2E . 2 . . . . 2 . 2 . . . . 1 . 2 . . . . 1 . 2 . . 65 . . 1 . 2 6 . 65 . . . 1 . 2 7 . . . 1 . . 7. . . 1 . . . 7 . . 1 . . .. . 60 . 1 1 . . 60% . . . 1 1 . . .. . . . 1 . . . . . . . 1 . . . . . . . 1 . . . . . 55 . . 1 . . . . 55 . . 1 1 . . . . . . 1 1 1 . . . . . . . 1 1 1 1 . . . . . .. 1 1 1 . . . . . 50* 1S1T A R T I N G . G U E S.S I N G . 50% 1 1 (Pupil) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 0 10 20 30 40 50 THE TOTAL NUMBER OF ANSWERS ENTERED SCORE ---------------------------------------------------------- Question File Numbers for the Above 50 Answers and the Number Answered in Each Pass Through the File ---------------------------------------------------------- 1 2 3 4 5 6 7 8 9 10 ---------------------------------------------------------- 0 15 46 24 35 22 4 26 25 32 2 P#1-----------<know and can do>----------------- 10 47 10 28 38 16 33 8 41 48 6 -------------------------------------------------- 20 14 1 45 44 9 5 17 37 12 49 -------------------------------------------------- 30 11 27 42 7 13 43 50 29 36 20 >(31) P#2------<educated guessing>--------------- 40 23 34 30 3 31 40 39 21 19 18 ------------------------->(15) P#6 P#7----->( 3) <random guessing> ---------------------------------------------------------- Task #1: Write and Enter Questions 5/13 The easiest and the most difficult of the five tasks is to create a question file. The easiest way is to copy from old tests or old test banks. The most difficult way is to create new questions. The original program was created to help rapidily produce high quality questions from current instructional materials. Eventually the method was recorded as a guide for students (see Critical Reading with Question Writing, pages 9-11). A QUESTION IS ENTERED IN PARTS. The minimum is the question stem and one answer (short answer) or two answers (multiple choice). There is space for two right and four wrong answers. Each file is limited to 200 questions. QUESTION FILE NAMES are limited to eight characters plus a three character extension (BIOLOGY1.TXT). Some thought should be given to how files are to be named. One method is to use a separate disk for each course, and name the files for text chapters or for unit tests. This TASK CAN BE INTERRUPTED at any point. On returning, the old file can be picked up again or the new questions can be appended to the old file. If the old file is picked up, the first screen shows the last question in the file. Questions in the file can be located by a FIND FUNCTION. The ERASE FUNCTION only marks a question for erasing. It can be retrieved by the Find function: (F erase) will display all items marked for erasing. This task was specifially designed for COPYING OLD QUESTIONS. It can be assigned to anyone with a minimum of instruction (see Problems, page 11). When the end of space is near, each additional character produces a beep. When CREATING QUESTIONS, the task allows one to enter questions as they occur, in incomplete fragments, which can be completed later. The next task, Select and Edit, has even greater flexibility for creating and proofing questions after the initial file is created with this task. Review and Test can also be used to view questions. However, it does not permit editing questions. Instead, make notes of changes, and make them with task #2, Select and Edit. In general, the function to be repeated in a task is assigned to the <RETURN> key. This increases productivity once you have become familiar with the task. Task #2: Select and Edit Questions 6/13 This is my favorite. Originally it was used to select questions from several question (data) files into a pool. Then several different versions were formated from the pool. STUDENT: Create by writing question stems that describe what you know or can do. Your understanding. Add a key term or phrase for a right answer. Add closely related unacceptable terms for wrong answers. TEACHER: As an assessment, score with the above rubric. For free- and forced-choice tests, select and edit from student created review question files. Start assignments with a file of descriptions that students then complete and put in order. Change the order of the questions with MOVE. New questions are added at the end of the file. Place them where you want them with MOVE. Again the <RETURN> key is assigned to a repetitive function: PASSING. Each tap of the key shows the next question in the file. The SELECT FUNCTION places a question into the new file of pooled questions. The REJECT FUNCTION drops it from being viewed again. When working with high quality question files, the automatic feature saves much time. For example, a 50 question test pool can be drawn 10 questions each from 5 files at random in a few minutes. This test then becomes a sub-sample of all of what could have been asked. It is a sub-sample of what students were expected to know or be able to do. The score has predictive power at the formal (higher) level of thinking. When such a test is administered as so students can report what they know or can reason, rather than as a typical multiple guess test (the concrete and random quessing levels of thinking), three levels of thinking can be extracted with an appropriate scoring, item analysis, and grading program: TRAINER. Also the quality of the questions can be determined statistically if more than 20 (ideally 50 or more) answer sheets are pooled. Question quality can also be determined by inspection. Are there two good acceptable answers? Do the wrong answers include options commonly associated with the stem or that are closely related? (See Critical Reading with Question Writing, pages 9-11.) Task #3: Print Cram Notes and Files 7/13 Most students and many faculty do not have their own computers or even ready access to one. Printed data files can be edited anywhere and at any time by faculty and students. The DATA FILE PRINTOUT can be used at the formal level of thinking as a study aid. For the concrete level of thinking, select the CRAM NOTE function. Save a tree! Both printouts use much less paper than listing the actual data file. Also each question part is labeled to match the editing prompts. Now that you are printing, you need to be aware of current copyright LAWS and the ETHICS of proper citation. Always cite the source on the Comment line when writing a question file. This is not only proper, but keeps your records in order. Current COPYRIGHT LAW states that copyright exists as the result of creation. Therefore making more than one copy for personal study purposes requires permission from the original author. If, however, you have created the questions, you are the original author. If you are COPYING from a published test bank (teachers can do this if they are using the text in their classes), say so. If you are copying from an old test, cite the teacher and the test: Mr. Jones, General Biology, Test 3. These questions are for your own personal use, no sharing, no duplicating. Likewise, if you are CREATING your own questions, which does not require permission to copy, again cite the related study material. The related study material can be an old test, a textbook, class notes, or from other reading or observational assignments. These questions can be shared, duplicated and sold, by your permission. For example, the instructions for the Missouri Driver's Test. If you turn this into a series of related questions in a rational order, you have a product for distribution to your classmates. (See Critical Reading with Question Writing, pages 9-11.) To create a question file that is not a copy of another test, follow these steps: 1. Mark out all words except the key terms. 2. Classify the key terms in a question. 3. Write new key terms that fit the same classification. Generic action and relationship key terms can be reused. 4. Write your question from your key terms. Task #4: Format and Print Tests 8/13 This task transforms a data file into a text file for printing out on paper. If the questions have been pooled, all that remains to do is to name the data file, select automatic, and the number of items. In a few seconds, a new test is created ready for printing. However, many faculty members like to pick and edit each question individually (concretely). This task can be performed by drawing questions from one or more data files. As each question is presented, it can be selected, rejected, edited, or passed. If selected, the formatted question can also be selected, rejected, edited or passed. Editing the DATA base makes a permanent change, editing the formatted item changes the question for the current test only. A record is kept of the number of questions selected from each data file. As the program ends, you will be asked to confirm if changes to the data base are to be made permanent. When finished selecting, the formated questions are printed to a file followed by the answer key. This file can be printed out as the test or picked up with a word processor to add a heading, instructions and illustrations. Each time a question file is formatted into test questions, the answers are randomized. Problem questions generate new answer options. Each formatting produces a different practice or real paper test. One or both of the following should print the formatted file (assuming the file is in drive A, and you replace File.Nam with the name of your test file): 1. A> COPY File.Nam PRN: 2. A> TYPE File.Nam > PRN The pooled data file can also be picked up with Review, Game and Test and used as a practice or makeup test. Critical Reading with Question Writing 9/13 (A Textbook and Course Supplement) -- INSTRUCTIONS -- STEP I: Label Listing (Preparing for a Mental Adventure) A. Look at the assignment. List words and phrases you feel are important for you to make sense of the material. You do not need to know the meaning of these words and phrases at this time. B. Make two lists of words and phrases that are important to you: (1) These I know or can do and (2) These I do not know or can not do. STEP II: Questioning (Meaning Making with Relationships) (and Label Descriptions) C. Now return to the beginning of the assignment. D. Read until you find or need a description. E. Complete a question (on a paper blank or the computer). 1. Question stem (A description in your words. What you know or do, what makes sense.) 2. Right answer(s) (An important word or phrase. A label for the above description.) 3. Wrong answer(s) (Word or phrase you associate with the question or right answer.) F. Continue D and E until all terms on your two lists have been used. That is, have been described in a question stem or used as answers. This is your report of what in your judgment needs to be mastered to understand or do the assignment. Additional questions can be created by using relationships and applications as question stems. G. The last step in writing an review question file is editing: 1. The author edits to correct any spelling or other grammatical errors. When writing in a group, each member is responsible for questioning the report. 2. The reviewer (who did not write the report) edits to add, remove, or change questions to match his/her understanding and course requirements. Question Form 10/13 A. Question (stem or statement or problem) Examples: The top of a road map is the direction: ________ The direction ( ) is at the top of road maps. _ | | | | | | | | | | | | | | |__________________________________________________________| B. RIGHT answer __ Example: north _______________________________________ | | | | | | |__________________________________________________________| C. RIGHT answer (alternate label, synonym or skip) __ Example: north terrestrial pole ______________________ | | | | | | |__________________________________________________________| D. WRONG answer (antonym, classification, description or label related to the right answer or to question stem) __ Example: south _______________________________________ | | | | | | |__________________________________________________________| E. WRONG answer (or skip) __ Example: down ________________________________________ | | | | | | |__________________________________________________________| F. WRONG answer (or skip) __ Example: left ________________________________________ | | | | | | |__________________________________________________________| G. WRONG answer (or skip) __ Example: varies with the map company _________________ | | | | | | |__________________________________________________________| (Use blank sheets for questions following this question.) Problems 11/13 Questions that require a numerical calculation can have many different answers if question and answer codes are used in the question. All values except one are held constant. The variable value is given as a range. Each time the question appears, the variable value is selected at random and new answer options are presented. Example: Q1 A wave with a frequency of 10 hertz is moving how fast if the wavelength is **2-100 m? R1 **10 m/s R2 W1 */10 m/s W2 /*10 m/s W3 *+10 m/s The question variable range code (**##-##) is **2-100. Values from 2 through 100 will be selected at random and then modified by each answer code. In the example, the constant is 10 from 10 hertz. The constant can be derived from more than one value in the question (as from a formula). Answer codes: ** variable * constant multiply */ variable / constant divide /* constant / variable divide *+ variable + constant add *- variable - constant subtract -* constant - variable subtract (The problem and resulting question can be viewed together with Select and Edit, Task #2, of Create and Edit.) Nonrandomizing Answers To make True Multiple-Choice compatible with traditional multiple-choice questions furnished in test banks, the following limitations have been made: True and False True always prints before false All of the above One or both options print last and None of the above in this order **L as first answer L is the location (A - E) of the right answer in a fixed set of nonrandomizing answers Asking the Questions (1980-1990) 12/13 Hart, Richard A. 1981. Evaluating and rewarding student initiative and judgement or an alternative to "sitting through" a course if you did not test out. Pages 75-76 in Directory of Teaching Innovations in Biology. Meeth, L. R. and Dean S. Gregory, Ed. Studies in Higher Education:Arlington, Virginia. 252 pages. Hart, Richard and Kenneth Minter. 1985. Using a computer to manage typical classroom problems. National Science Teachers Association Annual Meeting, Cincinnati, Ohio 18-21 April. Minter, Kenneth and Richard Hart. 1986. Essay testing using multiple choice questions. Missouri Academy of Science Annual Meeting, Warrensburg, MO 25-26 April. Hart, Richard and Kenneth Minter. 1988. Diagnostic Testing Using Multi-Choice and Matching Questions. National Science Teachers Association Annual Meeting, St. Louis, MO 7-10 April. Minter, Kenneth and Richard Hart. 1989. Student Choice in Computer Graded Tests. National Science Teachers Association Annual Meeting, Seattle, Washington 6-9 April. Hart, Richard. 1989. Practical Programs for Oversized Classes Resulting from Investigations into the Use of a Campus Computer System as a Part of an Instructional System (1981-1989). 217 pages. TX 2 602 807 Hart, Richard. 1989. Professor of the Year. Northwest Missouri State University. Hart, Richard. 1990. Professor of Biology, Emeritus, Northwest Missouri State University. The Answers (1991-1994) Hart, Richard. 1991. Founded Nine-Patch Software to underwrite the distribution of research results. (Nine-patch is the quilt design created from the three levels of thinking used by the three parties involved in education: students, teachers, and administrators. The nine squares represent the nine ways anything said or done in a classroom can be interpreted.) Hart, Richard and Kenneth Minter. 1991. Student Choice in Multiple-Choice Testing. National Science Teachers Association Annual Meeting, Houston, Texas 27-30 March. (continued) 13/13 Hart, Richard. 1992. Publication of quality scoring research results in three practical computer DOS programs. TRAINER for class tests, EQUALIZER for individual interactive use, and OLYMPIAD, a specific contest scoring application of TRAINER. Hart, Richard. 1993. Founding board member and volunteer treasurer of the Educational Software Cooperative, Inc. The ESC publishes educational software on CD-ROM and assists authors and teachers on CompuServe, Edforum, Section 2. Hart, Richard. 1994. EQUALIZER (DOS) edited into EXPEDITOR (VB for DOS and Windows). Application in the Classroom (1995-) Motes, Andrew. 1995. ESC founder, features both traditional forced-choice scoring for ranking and free-choice scoring for knowledge and judgment in the most recent verions of the award winning children's educational software "Best of 1994", School-Mom Plus. West, Rosemary. 1995. ESC home page: http://execpc.com/~esc Hart, Richard. 1995. TRAINER (scores knowledge and judgment) and True Multiple-Choice (write, review, game and test) released for middle-school, high school and college. ------------------------------------------------------------ Quality Management (TQM, CQI) Knowledge and judgment (quantity and quality) test results are of instructional value in any classroom. They are also of administrative value, especially for a quality management system such as CQI (1), QE (2), TQM (3), classical OBE, New Standards, etc. 1. Hubbard, Dean L., Editor. 1993. Continuous Quality Improvement. Maryville, Missouri: Prescott Publishing Co. 524 pages. 2. Rinehart, Gray. 1993. Quality Education. Milwaukee, Wisconsin: ASQC Quality Press. 328 pages. 3. Tribus, Myron. 1993. TQM in Education: The Theory and How To Put It To Work. ERIC Accession Number ED370168. 22 pages. ------------------------------------------------------------